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Two student populations of 13-year-olds in Ontario 
(Canada) were assessed as part of the 1988 International Assessment 
of Educational Progress (lAEP) : French- and English-language 
students. Results confirm findings of differences in mathematics 
achievement seen earlier in tht Second International Mathematics 
Study. Ontario French-language students performed poorly in relation 
to students in the other Canadian provinces. Unlike the earlier 
study, however, problems with translation and generalizability of 
results were not apparent. Analysis of the item-by-item data 
identified mathematics and science achievement differences between 
the French- and English-language populations in Ontario and between 
Ontario French-language and Quebec French-language students. The item 
response theory scaling procedure made presentation of both the 
results and the meaning of these re .its accessible to policymakers, 
educators, and the public. The complementary nature of the lAEP and 
the Ontario Provincial Review Program is noted, and recent 
initiatives by the Ontario Ministry of Education addressing 
French-language education are discussed. Two bar graphs and one data 
table are included. (Author/TJH) 
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Abstract 



Ontario assessed two student populations of 13 year olds as part 
of lAEP: French-language and English-language students. Results 
confirmed findings of differences in mathematics achievement see 
earlier in the Second International Mathematics Study. Unlike 
the earlier study however, problems with translation and 
generalizability of results were not apparent. 

Analysis of the item by item data identified mathematics and 
science achievement differences between the French and English- 
language populations in Ontario and between Ontario French- 
language and Quebec French-language students. The IRT scaling 
procedure made presentation of both the results and the meaning 
of these results accessible to policy makers, educators, and the 
public. 

The _ complementary nature of the lAEP with the Ontario Provincial 
Review Program was discussed and recent initiatives by the 
Ontario Ministry of Education which address French-language 
education were presented. 



ERIC 



3 



The Ontario Education System 



Ontario, with a population of approximately 9 million has the 
largest population of the ten Canadian provinces. Almost two 
million children (1,878,000) attend school and there are 
approximately 102,000 13-year-olds. 

Enrollments in publicly funded schools (public and Roman Catholic 
Separate) account for 97 percent of the school population for 
ages 6 to 16. Catholic schools educate about one-third of these 
students. Five percent of the population ages 6 to 15 attends 
French-language schools, and the great majority of these schools 
are part of the Separate school system. 

In Canada, education is a provincial responsibility. The Ontario 
Ministry of Education issues official curriculum guidelines and 
lists of approved textbooks but the system allows significant 
discretion to local school boards to determine specific 
curriculum content and instructional methodology. 

No formal streaming or tracking is provided through eighth grade, 
but students are expected to choose either the advanced 
(university-bound) , general, or basic level for their ninth-grade 
courses. Most students in grade 8 are found in K-8 elementary 
schools. Students can obtain a secondary-school diploma following 
completion of 30 credits. Each credit involves 100 to 120 hours 
of instruction; 16 credits are compulsory for most students. 
Students wishing to attend university must take six credits at 
the academic course level (formerly grade 13) . 



Provincial Assessment 

Prior to 1968, the Ministry of Education administered 
province-wide examinations as a basis for awarding the grade 13 
diploma. These examinations were replaced in 1968, for 
postsecondary admission purposes, by the Ontario Test for 
Admission to College and University. These tests were 
subsequently discontinued in 1974. 

Since the ending of these assessments there has been increasing 
concern about standards in the province. Most recently two 
government-initiated reports were critical of achievement levels 
obtained by students (e.g. Radwanski, 1988) and a Select 
Committee of the Ontario Legislature has been established to 
examine the goals and purposes of education. These increasing 
concerns about levels of achievement contributed to Ontario's 
involvement in the lEA mathematics and science studies and the 
lAEP involvement. It was during the lEA mathematics study that 
the Franco-Ontarian achievement issue began to emerge. 
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The SIMS and Franco-Ontarian Achievement 

Ontario participated in the lEA Second International Mathematics 
Study (SIMS) during the 1981-1982 school year. Ontario's sample 
at the Grade 8 level consisted of 130 schools of which ten were 
French-language. (This does not refer to French immersion classes 
which are considered English-language.) Ontario's Grade 12/13 
sample consisted of only 5 French-language secondary schools. 
(There are approximately 250 French-language elementary schools 
(K-8) and about 40 French-language secondary schools in 
Ontario . ) 

Student achievement of the French-language students in the 
Ontario Grade 8 sample, as compared to student achievement among 
English-language students, was poor (McLean, Raphael, and 
Wahlstrom, 1983) and is presented in the first figure. No 
reliable differences were seen at the secondary level between 
French and English language schools. 

While the French-language sample was small, the magnitude of the 
French-English language differences suggested some cause for 
concern. Before the final Ontario SIMS report was finished 
however, criticism was published concerning the quality of the 
translation (Hanna and Ladouceur, 1986) and though 130 of the 
180 items had not been criticized and some of the problems were 
relatively minor (McLean, Raphael, and Wahlstrom, 1985) , these 
translation concerns led to the comparative data being deleted 
from the final report (McLean, Raphael, and Wahlstrom, 1986) . 

[Simon (1987) conducted an elaborate analysis of item bias of 
the Ontario English and French-language data using five 
statistical techniques ~psuedo-IRT, Mantel-Haenszel, iterative 
logit procedure, partial correlation method and full chi-square 
approach — and found that while all the methods except the 
partial correlation significantly correlated with each other, 
results of these analyses did not agree in number, degree, or 
direction of item bias with the subjective judgments of bias made 
by experts . ] 

A generalizable sample of Frencn schools was not chosen in 
Ontario for the SISS. 



lAEP Involvement and the Examination of Ontario 
Student Achievement 

Ontario's involvement in the International Assessment of 
Educational Progress provided then, the first real comparative 
data concerning English and French-language student achievement 
m the province. (As will be noted later, subsequent data from 
the Ontario Provincial Reviews confirmed somewhat the English 
French-language performance difference in science.) 

Generalizable samples of both language groups were chosen in 
Ontario, New Brunswick, and Quebec, and the combined resources of 
the three provinces' Ministries of Education allowed for a "fool- 
proof" translation of the items into French. Every item was seen 
as relevant to the curriculum and these validity checks were 
extensive and repeated. The outcomes of the analyses were awaited 
with some anticipation. 



French and English-language lAEP Results in Ontario 

As data cleaning began in May 1988, preliminary analysis of the 
item by item data confirmed suspicions concerning French and 
English-language performance in Ontario. In mathematics, the 
mean difference across the initial 69 items in the assessment 
(these included some SIMS items) favored the English-language 
sample by 8.9 percentage points. English-language students 
outperformed French-language students on 64 items, the samples 
tied on one, and French-language students outperformed English- 
language students on four items. The largest of the French- 
language over English-language difference was five percentage 
points while 14 items favored the English-language sample by at 
least 15 percentage points. 

In science, the average English-language over French-language 
difference was 7.6 percentage points for the initial 66 items. 
English-language performance was higher for 54 items, while 
French-language scored higher on 12. The strongest French- 
language item favored students by 14 percentage points while 12 
items favored English-language students by at least 15 percenta< 
points. 

Another check of the translations by curriculum experts 
confirmed that the translations were accurate and were probably 
not the source of the French English-language differences in 
achievement. Additionally, hypotheses were advanced concerning 
the causes of these discrepancies. Some of these reasons are 
outlined in subsequent sections. 
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French and English-language Results in the Other Provinces 

The receipt of the item by item results from the other Canadian 
provinces in September 1988 confirmed that Ontario French- 
language students had indeed performed poorly in relation to 
students in the other Canadian provinces. (Much solace was not 
taken m knowing that Ontario French-language students had 
performed as well as American students.) At the Ministry of 
Educ-tion actions were formulated to deal with the results as the 
results of the IRT were awaited. 



The IRT Scaling and the lAEP Final Report 

By the time the results of the final iRT scaling were received 
the scope of the problem had been seen and the production of 
overall proficiency scores provided a means of presenting the 
results (the mathematics results are presented in this paper) in 
a clear, concise manner to the policy makers, educators, and f e 
public. While the rankings (Figure 1.1) basically replicated the 
mean percentage correct analyses, the proficiency levels and the 
percentages of students associated with each (Table 1.1) were 
particularly helpful in describing what these results meant. 



Follow-Up and the Identifying of Specific Problem Areas 

The follow-up to the lAEP involvement involves a variety of 
government initiatives and the use of the Provincial Review 
Program to focus on aspects of French-language education. 



Government Initiatives 

Ontario's commentary in the international report, A World of 
Differences (Lapoint, Mead, and Philips, 1989) pointed out that: 
"Financial and legal recognition of French-language schools has a 
short history in Ontario. There continues to be a shortage of 
French-language educational resources and the relatively small 
percentage (5%) and the wide dispersion throughout the province 
of the francophone community contributes to the difficulty of 
meeting the educational commitment to French-languaae education." 



The Ministry has announced the following initiatives related to 
French-language education: 

Renouveau en enseignement de math6matiques (REMA) and Renouveau 
en science 616mentaire (RESEL) are ministry-supported projects 
designed to provide French-language elementary school teachers 
with local professional development and curriculum resources in 
mathematics and science. 
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The Ministry's French-Language Fund for the Development and 
Production of Learning Materials is giving priority to resources 
m the areas of mathematics and science. The same applies for 
the development of computer software. 

Recent legislation and Ministry policies have now given Franco- 
Ontarians more autonomy than previously in the identification of 
their needs and in the decision -making with regard to the most 
efficient use of their allocated resources. 

Additionally, there are plans to develop an action plan in 
consultation with the French-language associations and faculties 
of education for systematic focussed teacher training and 
development of materials in mathematics and science. 



Provincial Review Program 

In 1987, Ontario initiated a provincial review process which 
provides information on curriculum implementation and student 
achievement (Raphael, 1987) . Through the use of school-based 
multiple matrix item sampling, provincial levels of student 
achievement are obtained in the areas of science, mathematics, 
and first language. 

Teacher and school information, including student 
opportunity-to-learn data, are also collected. A pilot review of 
?oof^^^" Geography at grades 9 and 10 was conducted in 

1S86-1987 (Cussons, Morrow, and Raphael, 1988 a,b) , and grades 11 
and 12 advanced-level Chemistry and Physics followed in 

1987- 1988 (McTavish, Morrow, and Raphael, 1989 a,b; 
Wrigglesworth, Morrow, and Raphael, 198 9 a,b) . During the 

1988- 198 9 school year, provincial reviews of grade 6 Reading and 
Mathematics are being conducted. The review process operates on a 
five-year cycle, assessing one subject area at three levels each 
year. Samples representative of both English and French-language 
students are chosen. Boards have the option of joining the 
review and these boards receive detailed reports as does each 
participating school. 

In the recent Senior Division Chemistry and Physics review 
(grades 11 and 12) reliable French and English differences 
emerged similar in magnitude to that seen in lAEP (McTavish, 
Morrow, and Raphael, 198 9 a,b; Wrigglesworth, Morrow, and 
Raphael, 1989 a,b) . However, since the provincial review uses a 
greater number of items then did lAEP (about 100 in each area of 
Chemistry and Physics, and five extended response questions 
requiring elaborated answers and work in each subject) and 
collects very detailed teacher and school level data, more 
elaborate analysis can identify possible sources of the English 
French-language differences. No scaling is done of these data. 

5 



ERIC 



S 



lAEP and Provincial Reviews 

These provincial reviews he wever do not provide comparative data 
from other provinces and are relatively expensive and elaborate 
undertakings. The reaction to the lAEP comparative French 
language data from other Canadian Provinces has stimulated 
discussion and reflection within the province. 

These projects then can be seen as complementary. The lAEP 
provides reliable and valid data concerning system functioning 
lAEP provides an impetus, as do the provincial reviews, for 
discussion, analysis, and if necessary, reform of educational 
practice m the province. lAEP findings promote support for the 
purposes of the newly developed provincial review program, in the 
case of French-language education it provides support for recent 
government initiatives. 

Analytically, the Provincial Reviews delve further into the 
sources of these findings. Reviews are more encompassing and 
collect a variety of data at different levels: school, board, and 
province. 



The Future of Irterprovincial and International Assessments 

Ontario expects to continue its involvement with lAEP. Ontario 
IS also involved with the other Canadian provinces and 
territories in a program of developing Canadian national 
indicators as part of a initiative by the Council of Ministers of 
Education, Canada and is continuing its involvement with the OECD 
Educational Indicators Project. 
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